The present invention relates to an apparatus for synchronized playback of audio-video (hereafter abbreviated as AV) signals, wherein compressed digital data of AV signals are decompressed into a video signal component and an audio signal component to be represented in synchronization with each other.
In the synchronized playback of AV signals, compressed digital data of AV signals read out from recording media such as a CD-ROM (Compact Disk Read Only Memory) or a hard disk device are decompressed before being represented through output devices such as a CRT monitor and a speaker. When the apparatus for the synchronized playback is provided with a data processing ability sufficiently high for the data decompression of the AV signals, or when the video signal size of the compressed digital data is comparatively small, the data decompression of both of the video signal component and the audio signal component can be performed within a short time, and so, audio data and video frames can be reproduced without interruption.
However, when the data processing ability of the apparatus is not sufficiently high compared to the video signal size of the compressed digital data, the apparatus can not reproduce both the audio signal component and the video signal component in real time with which they are recorded originally, taking too much time for the data decompression processes. As a consequence, the video signal component may be represented as a slow-motion film and the audio signal component is reproduced intermittently.
The intermittent reproduction of the audio signal component gives a sense of intense incongruity to the human hearing. Therefore, the audio signal component is usually processed to be reproduced continuously with a priority over the video signal component. The data decompression and displaying process of the video signal component is performed when the audio signal component is not processed. Here, adequate thin down of the picture frames (so-called frame omission) becomes necessary for synchronizing the presentation timings of the audio signal component and the video signal component so that the video signal component may not be delayed from the audio signal component.
As for the digital data compression, there is known an MPEG (Motion Picture Experts Group) standard, wherein the audio signal component and the video signal component separated of the AV signals are compressed independently into two data streams, an audio stream and a video stream. Each of the two data streams is divided into blocks, called packets, having a fixed length generally determined according to requirement of recording or transmission media, that is, a sector length of CD-ROM, for example. The audio stream and the video stream are multiplexed into a system stream composed of a mixed succession of these packets ranged in time order of both the audio stream and the video stream.
FIG. 2 is a schematic diagram illustrating a system stream 201 comprising video packets V1 to Vm composing a video stream 202 and audio packets A1 to An composing an audio stream 203.
Apart from the unit of packets, the video stream 202 consists of compressed data of each picture frame denoted by a symbol I (Intra picture), B (Bidirectionlly predictive picture) or P (Predictive picture) as illustrated in FIG. 2, while the audio stream 203 consisting of compressed data of each audio frame denoted by a symbol A. Reproduction time of each picture may differ to that of each audio frame. Therefore, timing information called a time stamp is comprised in header information of packets including beginning of a picture or an audio frame, so as to enable synchronized playback of the AV signals referring thereto.
As a prior art technique of the synchronized playback of the audio (voice) signal component and the video (picture) signal component from such compressed data, there is, for example, a Japanese patent application entitled "An apparatus for compression and reproduction of AV signals" and laid open as a Provisional Publication No. 107514/'95 (hereafter called the first prior art apparatus).
In the first prior art apparatus, color difference information is thinned down in a video data compression procedure considering the processing speed of the reproduction apparatus. Compensating the thinned down color difference information by simple interpolation, the decompression procedure of the video data is performed at a high speed, enabling synchronized reproduction of the video data and the audio data.
In another Japanese patent application entitled "Variable speed synchronized playback of recorded digital audio and video data" and laid open as a Provisional Publication No. 303240/'95 (hereafter called the second prior art apparatus), there is proposed another method of synchronized playback with a variable speed of the audio signal component and the video signal component.
In this method, a time required for decompression and playback of a frame of the video signal component, and a time required for decompression and reproduction of a block of the audio signal component are calculated from respective attribute information of each thereof. One of the calculated times is used as a master time clock for decompession of the audio signal component and the video signal component. The audio signal component and the video signal component are decompressed more speedily or more slowly according to a scaling factor of the master time clock indicated by a user.
There are also Japanese parent applications laid open as Provisional Publication No. 87324/'95 and No. 121276/'94 (hereafter called the third prior apparatus). In the third prior apparatus, the video signal component of a picture frame is compressed together with the audio signal component corresponding to the picture frame, and when data compression procedure of the video signal component is found to be delayed from that of the audio signal component, data compression of a part or a whole of the picture frame is omitted. In the same way, when data decompression of the video signal component is found to be lagged from that of the audio signal component in the reproduction, decompression of a part or a whole of the picture frame is omitted.
Thus, a synchronized reproduction of the AV signals is realized. Furthermore, having such a data stream as above mentioned, reproduction from a desired part of the AV signals can be performed in synchronization, in the third prior apparatus.
However, there are problems in these prior apparatus.
The problem of the first prior art apparatus is that feedback control means for the synchronization are not provided. In the first prior art apparatus, the thinning down of the color difference information is performed a priori in the data compression procedure according to the information amount of the video signal component to be processed, and no means are provided for detecting lead or lag of a displayed video frame to the corresponding audio data. Therefore, unnecessary thinning down of color difference information is performed even when the playback apparatus is provided with sufficient data processing ability, and restoration of the lead or lag of the video signal component to the audio signal component can not be performed flexibly, in the first prior art apparatus.
The problem of the second prior art apparatus is that a TDHS (Time Domain Harmonic Scaling) system is used for synchronizing reproduction of the audio signal component to the scaled master time clock. The TDHS system requires complicated processing which is inconvenient to be applied to the playback system with low data processing ability, without saying of the distortion of the audio signal component.
The problem of the third prior art apparatus lies in that they cannot be applied to compressed data having such a data stream as the MPEG standard wherein data of picture frames and audio frames having different reproduction time are compressed independently.
Furthermore, when the playback apparatus is implemented with software operating on an operating system with a low resolution software timer, there is a problem that correct synchronization between the audio signal component and the video signal component cannot be obtained because of the low resolution of the software timer, even if the feedback control means are provided. A fine resolution of a hardware timer, a CPU clock, for example, can be used, of course. However, use of the hardware timer makes the apparatus hardware-dependent, and thus not applicable to any device but to computers having a specific CPU clock hertz.
This is another problem of the prior art apparatus.